@dataknut)Please note that authorship is alphabetical. Contributions are listed below - see github for details and who to blame for what :-).
@dataknut)If you wish to refer to any of the material from this report please cite as:
Report circulation:
Report purpose:
This work is (c) 2020 the University of Southampton.
Data downloaded from http://southampton.my-air.uk. See also https://www.southampton.gov.uk/environmental-issues/pollution/air-quality/.
Southampton City Council collects various forms of air quality data at the sites shown in 2.1. WHO publishes information on the health consequences and “acceptable” exposure levels for each of these.
lDT <- data.table::melt(origDataDT, id.vars = c("site", "dateTimeUTC"), measure.vars = c("co",
"no2", "nox", "oz", "pm10", "pm2_5", "so2"), value.name = "value" # varies
)
lDT[, `:=`(obsDate, lubridate::date(dateTimeUTC))]
# remove NA lDT <- lDT[!is.na(value)]
t <- lDT[!is.na(value), .(from = min(dateTimeUTC), to = max(dateTimeUTC), nObs = .N), keyby = .(site,
variable)]
kableExtra::kable(t, caption = "Dates data != NA available by site and measure", digits = 2) %>%
kable_styling()
| site | variable | from | to | nObs |
|---|---|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | no2 | 2016-01-26 11:00:00 | 2020-03-11 13:00:00 | 34980 |
| Southampton - A33 Roadside (near docks, AURN site) | pm10 | 2016-01-04 13:00:00 | 2020-03-11 12:00:00 | 33485 |
| Southampton - Background (near city centre, AURN site) | no2 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 29812 |
| Southampton - Background (near city centre, AURN site) | nox | 2016-01-01 00:00:00 | 2019-01-01 00:00:00 | 20026 |
| Southampton - Background (near city centre, AURN site) | oz | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 29490 |
| Southampton - Background (near city centre, AURN site) | pm10 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 26065 |
| Southampton - Background (near city centre, AURN site) | pm2_5 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 28273 |
| Southampton - Background (near city centre, AURN site) | so2 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 29306 |
| Southampton - Onslow Road (near RSH) | no2 | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 35439 |
| Southampton - Onslow Road (near RSH) | nox | 2016-01-01 00:00:00 | 2020-03-11 13:00:00 | 35443 |
| Southampton - Victoria Road (Woolston) | no2 | 2016-01-01 00:00:00 | 2020-03-11 08:00:00 | 27116 |
| Southampton - Victoria Road (Woolston) | nox | 2016-01-01 00:00:00 | 2020-03-11 08:00:00 | 27116 |
Summarise previously downloaded and processed data… Note that this may not be completely up to date.
t <- lDT[, .(mean = mean(value, na.rm = TRUE)), keyby = .(site, variable)]
kableExtra::kable(t, caption = "Mean values per site (NaN indicates not measured)") %>% kable_styling()
| site | variable | mean |
|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | co | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | no2 | 35.571547 |
| Southampton - A33 Roadside (near docks, AURN site) | nox | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | oz | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | pm10 | 18.928350 |
| Southampton - A33 Roadside (near docks, AURN site) | pm2_5 | NaN |
| Southampton - A33 Roadside (near docks, AURN site) | so2 | NaN |
| Southampton - Background (near city centre, AURN site) | co | NaN |
| Southampton - Background (near city centre, AURN site) | no2 | 29.445508 |
| Southampton - Background (near city centre, AURN site) | nox | 50.789064 |
| Southampton - Background (near city centre, AURN site) | oz | 40.877009 |
| Southampton - Background (near city centre, AURN site) | pm10 | 18.079098 |
| Southampton - Background (near city centre, AURN site) | pm2_5 | 12.009391 |
| Southampton - Background (near city centre, AURN site) | so2 | 2.970521 |
| Southampton - Bitterne | co | NaN |
| Southampton - Bitterne | no2 | NaN |
| Southampton - Bitterne | nox | NaN |
| Southampton - Bitterne | oz | NaN |
| Southampton - Bitterne | pm10 | NaN |
| Southampton - Bitterne | pm2_5 | NaN |
| Southampton - Bitterne | so2 | NaN |
| Southampton - Onslow Road (near RSH) | co | NaN |
| Southampton - Onslow Road (near RSH) | no2 | 42.379229 |
| Southampton - Onslow Road (near RSH) | nox | 87.529707 |
| Southampton - Onslow Road (near RSH) | oz | NaN |
| Southampton - Onslow Road (near RSH) | pm10 | NaN |
| Southampton - Onslow Road (near RSH) | pm2_5 | NaN |
| Southampton - Onslow Road (near RSH) | so2 | NaN |
| Southampton - Redbridge | co | NaN |
| Southampton - Redbridge | no2 | NaN |
| Southampton - Redbridge | nox | NaN |
| Southampton - Redbridge | oz | NaN |
| Southampton - Redbridge | pm10 | NaN |
| Southampton - Redbridge | pm2_5 | NaN |
| Southampton - Redbridge | so2 | NaN |
| Southampton - Victoria Road (Woolston) | co | NaN |
| Southampton - Victoria Road (Woolston) | no2 | 38.925457 |
| Southampton - Victoria Road (Woolston) | nox | 80.190670 |
| Southampton - Victoria Road (Woolston) | oz | NaN |
| Southampton - Victoria Road (Woolston) | pm10 | NaN |
| Southampton - Victoria Road (Woolston) | pm2_5 | NaN |
| Southampton - Victoria Road (Woolston) | so2 | NaN |
Table 3.1 gives an indication of the availability of the different measures.
In this section we present graphical analysis of the previoulsy downloaded data. Note this is just a snapshot of the data available.
yLab <- "Nitrogen Dioxide (ug/m3)"
dt <- lDT[variable == "no2"]
t <- dt[, .(mean = mean(value, na.rm = TRUE), sd = sd(value, na.rm = TRUE), min = min(value,
na.rm = TRUE), max = max(value, na.rm = TRUE)), keyby = .(site)]
kableExtra::kable(t, caption = "Summary of NO2 data") %>% kable_styling()
| site | mean | sd | min | max |
|---|---|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | 35.57155 | 24.80918 | 0.0 | 164.3 |
| Southampton - Background (near city centre, AURN site) | 29.44551 | 16.95406 | 2.3 | 144.2 |
| Southampton - Bitterne | NaN | NA | Inf | -Inf |
| Southampton - Onslow Road (near RSH) | 42.37923 | 21.17712 | 1.2 | 241.3 |
| Southampton - Redbridge | NaN | NA | Inf | -Inf |
| Southampton - Victoria Road (Woolston) | 38.92546 | 23.85054 | -1.3 | 287.5 |
Table 4.1 suggests that there may be a few (5) negative values. These are summarised in 4.2 while Figure 4.1 shows the availability and levels of the pollutant data over time.
t <- head(dt[value < 0], 10)
kableExtra::kable(t, caption = "Negative NO2 values (up to first 10)") %>% kable_styling()
| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Victoria Road (Woolston) | 2016-01-29 04:00:00 | no2 | -0.1 | 2016-01-29 |
| Southampton - Victoria Road (Woolston) | 2016-11-27 05:00:00 | no2 | -0.5 | 2016-11-27 |
| Southampton - Victoria Road (Woolston) | 2016-11-27 06:00:00 | no2 | -1.1 | 2016-11-27 |
| Southampton - Victoria Road (Woolston) | 2017-02-06 00:00:00 | no2 | -0.2 | 2017-02-06 |
| Southampton - Victoria Road (Woolston) | 2017-02-06 01:00:00 | no2 | -1.3 | 2017-02-06 |
t <- table(dt[value < 0]$site)
kableExtra::kable(t, caption = "Negative NO2 values (count by site)") %>% kable_styling()
| Var1 | Freq |
|---|---|
| Southampton - Victoria Road (Woolston) | 5 |
# dt,xvar, yvar,fillVar, yLab
p <- makeTilePlot(dt, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p
Figure 4.1: Nitrogen Dioxide data availability and levels over time
Figure 4.2 shows daily mean values for all sites over time and includes smoother trend lines for each site.
plotDT <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE)),
keyby = .(obsDate, site)]
p <- makeDotPlot(plotDT,
xVar = "obsDate",
yVar = "mean",
byVar = "site",
yLab = paste0("Mean daily ", yLab)
)
p <- p +
geom_smooth() + # add smoothed line
labs(caption = "Trend line = Generalized additive model (gam) with integrated smoothness estimation")
if(doPlotly){
p
plotly::ggplotly(p + xlim(xlimMinDate, xlimMaxDate)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.2: Nitrogen Dioxide levels, Southampton (daily mean
dt <- dt[dateTimeUTC > oneYearAgo]
t <- dt[value > hourlyNo2Threshold_WHO][order(-value)]
kableExtra::kable(caption = paste0("Values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
", last 12 months)"), head(t, 10)) %>% kable_styling()
| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
p <- makeDotPlot(dt[!is.na(value)], xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + geom_hline(yintercept = hourlyNo2Threshold_WHO) + labs(caption = "Reference line = WHO hourly threshold")
plotly::ggplotly(p) # for interaction
Figure 4.3: Nitrogen Dioxide levels, Southampton (hourly, last 12 months)
Figure 4.3 shows hourly values for all sites for the last 12 months. In this period there were 0 hours when the hourly Nitrogen Dioxide level breached the relevant WHO hourly threshold (200). The worst 10 cases (if any) are shown in Table 4.3.
Clearly there are winter peaks, the mean daily values show less variance (and less extremes) than the hourly data and there has also been a decreasing trend over time.
PM 10 data: has more sensors and wider coverage than PM2.5
yLab <- "PM 10 (ug/m3)"
dt <- lDT[variable == "pm10"]
t <- dt[, .(mean = mean(value, na.rm = TRUE), sd = sd(value, na.rm = TRUE), min = min(value,
na.rm = TRUE), max = max(value, na.rm = TRUE)), keyby = .(site)]
kableExtra::kable(t, caption = "Summary of pm10 data") %>% kable_styling()
| site | mean | sd | min | max |
|---|---|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | 18.92835 | 14.25055 | 0.0 | 761.6 |
| Southampton - Background (near city centre, AURN site) | 18.07910 | 11.32865 | -3.5 | 344.1 |
| Southampton - Bitterne | NaN | NA | Inf | -Inf |
| Southampton - Onslow Road (near RSH) | NaN | NA | Inf | -Inf |
| Southampton - Redbridge | NaN | NA | Inf | -Inf |
| Southampton - Victoria Road (Woolston) | NaN | NA | Inf | -Inf |
Table 4.4 suggests that there may be a few (19) negative values. These are shown in 4.5 while 4.4 shows data availability and PM 10 levels over time at each site.
t <- head(dt[value < 0], nrow(dt[value < 0]))
kableExtra::kable(head(t), caption = "Negative PM10 values - first 6") %>% kable_styling()
| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2016-04-16 10:00:00 | pm10 | -0.9 | 2016-04-16 |
| Southampton - Background (near city centre, AURN site) | 2016-06-12 06:00:00 | pm10 | -2.0 | 2016-06-12 |
| Southampton - Background (near city centre, AURN site) | 2016-06-12 07:00:00 | pm10 | -2.1 | 2016-06-12 |
| Southampton - Background (near city centre, AURN site) | 2017-06-11 03:00:00 | pm10 | -1.2 | 2017-06-11 |
| Southampton - Background (near city centre, AURN site) | 2017-06-30 11:00:00 | pm10 | -0.1 | 2017-06-30 |
| Southampton - Background (near city centre, AURN site) | 2017-07-04 02:00:00 | pm10 | -0.6 | 2017-07-04 |
p <- makeTilePlot(lDT[variable == "pm10"], xVar = "dateTimeUTC", yVar = "site", fillVar = "value",
yLab = yLab)
p
Figure 4.4: Availability and level of PM 10 data over time
plotDT <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE)),
keyby = .(obsDate, site)]
extremePm10Daily <- plotDT[mean > dailyPm10Threshold_WHO][order(-mean)]
kableExtra::kable(caption = paste0("10 highest values greater than WHO threshold (PM 10 > ",
dailyPm10Threshold_WHO , ")"),
digits = 2,
head(extremePm10Daily, 10)) %>%
kable_styling()
| obsDate | site | mean |
|---|---|---|
| 2016-08-09 | Southampton - A33 Roadside (near docks, AURN site) | 144.48 |
| 2017-06-19 | Southampton - A33 Roadside (near docks, AURN site) | 143.66 |
| 2017-05-05 | Southampton - A33 Roadside (near docks, AURN site) | 88.94 |
| 2016-09-19 | Southampton - A33 Roadside (near docks, AURN site) | 88.46 |
| 2016-11-30 | Southampton - A33 Roadside (near docks, AURN site) | 79.25 |
| 2016-03-12 | Southampton - A33 Roadside (near docks, AURN site) | 72.00 |
| 2017-01-22 | Southampton - A33 Roadside (near docks, AURN site) | 66.86 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 64.67 |
| 2019-02-27 | Southampton - A33 Roadside (near docks, AURN site) | 60.60 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 58.37 |
p <- makeDotPlot(plotDT,
xVar = "obsDate",
yVar = "mean",
byVar = "site",
yLab = paste0("Mean daily ", yLab)
)
p <- p +
geom_hline(yintercept = dailyPm10Threshold_WHO) +
geom_smooth() + # add smoothed line
labs(caption = "Trend line = Generalized additive model (gam) with integrated smoothness estimation\nReference line = WHO PM10 mean daily threshold")
if(doPlotly){
p
plotly::ggplotly(p + xlim(xlimMinDate, xlimMaxDate)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.5: PM10 levels, Southampton (daily mean)
nDaysOverThreshold <- uniqueN(extremePm10Daily$obsDate)
nDays <- uniqueN(plotDT$obsDate) # need to count days not site-days
Figure 4.5 shows daily values for all sites across the entire dataset and indicates the 19 days (1.3%) that breached the WHO PM10 daily mean exposure threshold (50) - see Table 4.6.
dt <- dt[dateTimeUTC > oneYearAgo]
t <- dt[value > 100][order(-value)]
kableExtra::kable(caption = "10 highest hourly values (PM 10 > 100)", head(t)) %>% kable_styling()
| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2019-10-27 20:00:00 | pm10 | 252.5 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 19:00:00 | pm10 | 174.6 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-10-27 19:00:00 | pm10 | 162.2 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 18:00:00 | pm10 | 114.3 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 21:00:00 | pm10 | 109.1 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-03-29 22:00:00 | pm10 | 102.8 | 2019-03-29 |
p <- makeDotPlot(dt[!is.na(value)], xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + labs(caption = "NB: There is no WHO PM10 hourly threshold")
if (doPlotly) {
p
plotly::ggplotly(p + xlim(xlimMinDateTime, xlimMaxDateTime)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.6: PM10 levels, Southampton (hourly, last 12 months)
Figure 4.6 shows hourly PM 10 values for all sites over the last 12 months and suggests there may be outliers (see Table 4.7).
yLab <- "PM 2.5 (ug/m3)"
dt <- lDT[variable == "pm2_5"]
t <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE), sd = sd(value, na.rm = TRUE), min = min(value,
na.rm = TRUE), max = max(value, na.rm = TRUE)), keyby = .(site)]
kableExtra::kable(t, caption = "Summary of pm2_5 data") %>% kable_styling()
| site | mean | sd | min | max |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 12.00939 | 9.47876 | -4 | 289.2 |
Table 4.8 suggests that there may be a few (89) negative values. These are shown in Table 4.9 while Figure 4.7 shows data availability and PM 2.5 levels over time at each site.
t <- head(dt[value < 0], nrow(dt[value < 0]))
kableExtra::kable(head(t), caption = "Negative pm2_5 values - first 6") %>% kable_styling()
| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2016-01-02 17:00:00 | pm2_5 | -0.8 | 2016-01-02 |
| Southampton - Background (near city centre, AURN site) | 2016-01-04 03:00:00 | pm2_5 | -0.9 | 2016-01-04 |
| Southampton - Background (near city centre, AURN site) | 2016-01-04 23:00:00 | pm2_5 | -0.3 | 2016-01-04 |
| Southampton - Background (near city centre, AURN site) | 2016-01-05 01:00:00 | pm2_5 | -1.0 | 2016-01-05 |
| Southampton - Background (near city centre, AURN site) | 2016-01-09 22:00:00 | pm2_5 | -0.3 | 2016-01-09 |
| Southampton - Background (near city centre, AURN site) | 2016-01-09 23:00:00 | pm2_5 | -0.9 | 2016-01-09 |
p <- makeTilePlot(dt, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p
Figure 4.7: Availability and level of PM 10 data over time
plotDT <- dt[!is.na(value), .(mean = mean(value, na.rm = TRUE)),
keyby = .(obsDate, site)]
extremePm25Daily <- plotDT[mean > dailyPm2.5Threshold_WHO][order(-mean)]
kableExtra::kable(caption = paste0("6 highest values greater than WHO threshold (PM 2.5 > ",
dailyPm2.5Threshold_WHO , ")"),
digits = 2,
head(extremePm25Daily)) %>%
kable_styling()
| obsDate | site | mean |
|---|---|---|
| 2016-03-12 | Southampton - Background (near city centre, AURN site) | 64.79 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 49.04 |
| 2016-03-13 | Southampton - Background (near city centre, AURN site) | 47.79 |
| 2019-04-07 | Southampton - Background (near city centre, AURN site) | 46.90 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 43.64 |
| 2019-03-29 | Southampton - Background (near city centre, AURN site) | 43.60 |
p <- makeDotPlot(plotDT,
xVar = "obsDate",
yVar = "mean",
byVar = "site",
yLab = paste0("Mean daily ", yLab)
)
p <- p +
geom_hline(yintercept = dailyPm2.5Threshold_WHO) +
geom_smooth() + #add smoothed line
labs(caption = "Trend line = Generalized additive model (gam) with integrated smoothness estimation\nReference line = WHO daily PM2.5 threshold")
if(doPlotly){
p
plotly::ggplotly(p + xlim(xlimMinDate, xlimMaxDate)) # interactive, xlimited # interactive
} else {
p
}
Figure 4.8: PM2_5 levels, Southampton (daily mean)
nDaysOverThreshold <- uniqueN(extremePm25Daily$obsDate)
nDays <- uniqueN(plotDT$obsDate) # need to count days not site-days
Figure 4.8 shows daily values for all sites across the dataset and indicates that the WHO PM2_5 daily mean exposure threshold (25) was breached on 83 days (6.9 %). The 6 worst cases are shown in Table 4.10.
dt <- dt[dateTimeUTC > oneYearAgo]
t <- dt[value > 50][order(-value)]
kableExtra::kable(caption = "Extreme hourly values (PM 2.5 > 50, last 12 months, worst 6)", head(t)) %>%
kable_styling()
| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Background (near city centre, AURN site) | 2019-10-27 20:00:00 | pm2_5 | 239.1 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 19:00:00 | pm2_5 | 167.6 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-10-27 19:00:00 | pm2_5 | 140.4 | 2019-10-27 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 18:00:00 | pm2_5 | 109.9 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-12-02 21:00:00 | pm2_5 | 103.9 | 2019-12-02 |
| Southampton - Background (near city centre, AURN site) | 2019-12-03 01:00:00 | pm2_5 | 90.9 | 2019-12-03 |
p <- makeDotPlot(dt[!is.na(value)], xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + labs(caption = "NB: There is no WHO PM2.5 hourly threshold")
plotly::ggplotly(p) # for interaction
Figure 4.9: PM2_5 levels, Southampton (hourly, last 12 months)
Figure 4.9 shows hourly values for all sites for the last 12 months while Table 4.11 reports the 6 worst hours.
skimr::skim(origDataDT)
| Name | origDataDT |
| Number of rows | 262398 |
| Number of columns | 10 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| logical | 1 |
| numeric | 7 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: logical
| skim_variable | n_missing | complete_rate | mean | count |
|---|---|---|---|---|
| co | 262398 | 0 | NaN | : |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| no | 135024 | 0.49 | 26.32 | 40.01 | -5.0 | 4.6 | 13.1 | 31.8 | 746.3 | ▇▁▁▁▁ |
| no2 | 135051 | 0.49 | 36.75 | 22.47 | -1.3 | 20.2 | 32.8 | 48.7 | 287.5 | ▇▂▁▁▁ |
| nox | 179813 | 0.31 | 76.21 | 74.88 | -1.4 | 32.3 | 56.5 | 93.8 | 1431.8 | ▇▁▁▁▁ |
| pm10 | 202848 | 0.23 | 18.56 | 13.06 | -3.5 | 10.8 | 15.7 | 23.0 | 761.6 | ▇▁▁▁▁ |
| oz | 232908 | 0.11 | 40.88 | 23.14 | -0.2 | 24.2 | 41.7 | 56.7 | 174.1 | ▆▇▂▁▁ |
| pm2_5 | 234125 | 0.11 | 12.01 | 9.48 | -4.0 | 6.5 | 9.3 | 14.3 | 289.2 | ▇▁▁▁▁ |
| so2 | 233092 | 0.11 | 2.97 | 3.27 | -1.1 | 1.0 | 1.7 | 3.5 | 50.5 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
skimr::skim(lDT)
| Name | lDT |
| Number of rows | 1836786 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 7 | co: 262398, no2: 262398, nox: 262398, oz: 262398 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 1480235 | 0.19 | 38.45 | 46.01 | -4 | 12.6 | 27.1 | 49.7 | 1431.8 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
t <- lDT[variable == "no2"]
skimr::skim(t)
| Name | t |
| Number of rows | 262398 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 1 | no2: 262398, co: 0, nox: 0, oz: 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 135051 | 0.49 | 36.75 | 22.47 | -1.3 | 20.2 | 32.8 | 48.7 | 287.5 | ▇▂▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
t <- lDT[variable == "no2" & value > hourlyNo2Threshold_WHO][order(-value)]
kableExtra::kable(caption = paste0("Values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
")"), t) %>% kable_styling()
| site | dateTimeUTC | variable | value | obsDate |
|---|---|---|---|---|
| Southampton - Victoria Road (Woolston) | 2016-11-30 17:00:00 | no2 | 287.5 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 19:00:00 | no2 | 275.3 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 18:00:00 | no2 | 268.7 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 18:00:00 | no2 | 251.6 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 17:00:00 | no2 | 247.3 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2017-01-24 18:00:00 | no2 | 243.5 | 2017-01-24 |
| Southampton - Victoria Road (Woolston) | 2017-01-24 19:00:00 | no2 | 242.0 | 2017-01-24 |
| Southampton - Onslow Road (near RSH) | 2016-03-08 09:00:00 | no2 | 241.3 | 2016-03-08 |
| Southampton - Victoria Road (Woolston) | 2017-01-03 08:00:00 | no2 | 234.4 | 2017-01-03 |
| Southampton - Victoria Road (Woolston) | 2017-01-03 09:00:00 | no2 | 232.8 | 2017-01-03 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 19:00:00 | no2 | 226.6 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 16:00:00 | no2 | 222.6 | 2016-11-30 |
| Southampton - Victoria Road (Woolston) | 2016-12-29 09:00:00 | no2 | 218.7 | 2016-12-29 |
| Southampton - Onslow Road (near RSH) | 2016-03-08 08:00:00 | no2 | 216.0 | 2016-03-08 |
| Southampton - Victoria Road (Woolston) | 2017-01-05 20:00:00 | no2 | 214.3 | 2017-01-05 |
| Southampton - Victoria Road (Woolston) | 2017-01-24 20:00:00 | no2 | 212.7 | 2017-01-24 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 09:00:00 | no2 | 209.9 | 2016-11-30 |
| Southampton - Onslow Road (near RSH) | 2016-03-12 22:00:00 | no2 | 208.5 | 2016-03-12 |
| Southampton - Onslow Road (near RSH) | 2016-03-08 07:00:00 | no2 | 208.1 | 2016-03-08 |
| Southampton - Victoria Road (Woolston) | 2016-11-30 20:00:00 | no2 | 207.0 | 2016-11-30 |
| Southampton - Onslow Road (near RSH) | 2016-02-19 08:00:00 | no2 | 206.3 | 2016-02-19 |
| Southampton - Victoria Road (Woolston) | 2016-01-19 19:00:00 | no2 | 204.4 | 2016-01-19 |
| Southampton - Onslow Road (near RSH) | 2016-03-21 08:00:00 | no2 | 202.8 | 2016-03-21 |
t <- lDT[variable == "pm10"]
skimr::skim(t)
| Name | t |
| Number of rows | 262398 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 1 | pm1: 262398, co: 0, no2: 0, nox: 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 202848 | 0.23 | 18.56 | 13.06 | -3.5 | 10.8 | 15.7 | 23 | 761.6 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
kableExtra::kable(caption = paste0("PM 10 values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
")"), extremePm10Daily) %>% kable_styling()
| obsDate | site | mean |
|---|---|---|
| 2016-08-09 | Southampton - A33 Roadside (near docks, AURN site) | 144.48125 |
| 2017-06-19 | Southampton - A33 Roadside (near docks, AURN site) | 143.66000 |
| 2017-05-05 | Southampton - A33 Roadside (near docks, AURN site) | 88.94348 |
| 2016-09-19 | Southampton - A33 Roadside (near docks, AURN site) | 88.45625 |
| 2016-11-30 | Southampton - A33 Roadside (near docks, AURN site) | 79.25417 |
| 2016-03-12 | Southampton - A33 Roadside (near docks, AURN site) | 72.00000 |
| 2017-01-22 | Southampton - A33 Roadside (near docks, AURN site) | 66.86250 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 64.67143 |
| 2019-02-27 | Southampton - A33 Roadside (near docks, AURN site) | 60.59524 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 58.37083 |
| 2016-12-01 | Southampton - A33 Roadside (near docks, AURN site) | 54.80833 |
| 2017-01-26 | Southampton - A33 Roadside (near docks, AURN site) | 53.90000 |
| 2019-03-29 | Southampton - Background (near city centre, AURN site) | 53.69167 |
| 2017-12-19 | Southampton - Background (near city centre, AURN site) | 53.66250 |
| 2016-12-06 | Southampton - A33 Roadside (near docks, AURN site) | 53.45000 |
| 2019-04-17 | Southampton - A33 Roadside (near docks, AURN site) | 53.08750 |
| 2019-04-07 | Southampton - Background (near city centre, AURN site) | 52.94167 |
| 2018-04-21 | Southampton - Background (near city centre, AURN site) | 52.32083 |
| 2019-03-30 | Southampton - Background (near city centre, AURN site) | 51.66250 |
| 2019-04-22 | Southampton - Background (near city centre, AURN site) | 50.86250 |
| 2019-02-23 | Southampton - Background (near city centre, AURN site) | 50.21667 |
t <- lDT[variable == "pm2_5"]
skimr::skim(t)
| Name | t |
| Number of rows | 262398 |
| Number of columns | 5 |
| _______________________ | |
| Column type frequency: | |
| character | 1 |
| Date | 1 |
| factor | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| site | 0 | 1 | 22 | 54 | 0 | 6 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 1827 |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| variable | 0 | 1 | FALSE | 1 | pm2: 262398, co: 0, no2: 0, nox: 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 234125 | 0.11 | 12.01 | 9.48 | -4 | 6.5 | 9.3 | 14.3 | 289.2 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-07-02 | 43733 |
kableExtra::kable(caption = paste0("PM 2.5 values greater than WHO threshold (NO2 > ", hourlyNo2Threshold_WHO,
")"), extremePm25Daily) %>% kable_styling()
| obsDate | site | mean |
|---|---|---|
| 2016-03-12 | Southampton - Background (near city centre, AURN site) | 64.79167 |
| 2019-04-17 | Southampton - Background (near city centre, AURN site) | 49.04167 |
| 2016-03-13 | Southampton - Background (near city centre, AURN site) | 47.79167 |
| 2019-04-07 | Southampton - Background (near city centre, AURN site) | 46.90417 |
| 2019-02-27 | Southampton - Background (near city centre, AURN site) | 43.63810 |
| 2019-03-29 | Southampton - Background (near city centre, AURN site) | 43.59583 |
| 2019-04-22 | Southampton - Background (near city centre, AURN site) | 42.92500 |
| 2017-02-12 | Southampton - Background (near city centre, AURN site) | 42.24583 |
| 2018-04-21 | Southampton - Background (near city centre, AURN site) | 41.96250 |
| 2016-03-11 | Southampton - Background (near city centre, AURN site) | 41.80000 |
| 2019-04-16 | Southampton - Background (near city centre, AURN site) | 41.77500 |
| 2019-03-30 | Southampton - Background (near city centre, AURN site) | 40.80000 |
| 2017-12-19 | Southampton - Background (near city centre, AURN site) | 40.45833 |
| 2016-01-20 | Southampton - Background (near city centre, AURN site) | 39.66667 |
| 2017-09-26 | Southampton - Background (near city centre, AURN site) | 39.12083 |
| 2017-03-27 | Southampton - Background (near city centre, AURN site) | 38.15833 |
| 2019-04-08 | Southampton - Background (near city centre, AURN site) | 37.65833 |
| 2019-04-18 | Southampton - Background (near city centre, AURN site) | 37.00417 |
| 2019-02-15 | Southampton - Background (near city centre, AURN site) | 36.48571 |
| 2019-02-23 | Southampton - Background (near city centre, AURN site) | 35.80833 |
| 2018-05-08 | Southampton - Background (near city centre, AURN site) | 35.52917 |
| 2018-05-07 | Southampton - Background (near city centre, AURN site) | 35.44583 |
| 2017-02-13 | Southampton - Background (near city centre, AURN site) | 35.42083 |
| 2018-05-06 | Southampton - Background (near city centre, AURN site) | 34.07917 |
| 2016-03-17 | Southampton - Background (near city centre, AURN site) | 33.73333 |
| 2018-04-11 | Southampton - Background (near city centre, AURN site) | 33.45000 |
| 2019-01-24 | Southampton - Background (near city centre, AURN site) | 33.33750 |
| 2018-04-14 | Southampton - Background (near city centre, AURN site) | 33.20417 |
| 2018-12-27 | Southampton - Background (near city centre, AURN site) | 33.03333 |
| 2018-11-22 | Southampton - Background (near city centre, AURN site) | 32.90833 |
| 2017-02-11 | Southampton - Background (near city centre, AURN site) | 32.72917 |
| 2017-04-08 | Southampton - Background (near city centre, AURN site) | 32.65833 |
| 2019-12-02 | Southampton - Background (near city centre, AURN site) | 32.61667 |
| 2020-01-21 | Southampton - Background (near city centre, AURN site) | 32.60417 |
| 2016-03-23 | Southampton - Background (near city centre, AURN site) | 32.31667 |
| 2018-11-23 | Southampton - Background (near city centre, AURN site) | 32.30417 |
| 2019-12-03 | Southampton - Background (near city centre, AURN site) | 32.15417 |
| 2018-05-26 | Southampton - Background (near city centre, AURN site) | 32.12083 |
| 2017-11-03 | Southampton - Background (near city centre, AURN site) | 31.53333 |
| 2019-04-15 | Southampton - Background (near city centre, AURN site) | 31.52917 |
| 2017-04-09 | Southampton - Background (near city centre, AURN site) | 31.25417 |
| 2019-04-23 | Southampton - Background (near city centre, AURN site) | 31.11250 |
| 2016-01-01 | Southampton - Background (near city centre, AURN site) | 30.73750 |
| 2017-03-24 | Southampton - Background (near city centre, AURN site) | 30.70833 |
| 2019-02-24 | Southampton - Background (near city centre, AURN site) | 30.55417 |
| 2018-11-24 | Southampton - Background (near city centre, AURN site) | 30.48333 |
| 2017-09-27 | Southampton - Background (near city centre, AURN site) | 30.29583 |
| 2018-04-12 | Southampton - Background (near city centre, AURN site) | 30.21667 |
| 2017-11-02 | Southampton - Background (near city centre, AURN site) | 30.10833 |
| 2019-03-31 | Southampton - Background (near city centre, AURN site) | 29.72500 |
| 2019-10-27 | Southampton - Background (near city centre, AURN site) | 29.64167 |
| 2018-10-05 | Southampton - Background (near city centre, AURN site) | 29.54167 |
| 2019-02-25 | Southampton - Background (near city centre, AURN site) | 29.40000 |
| 2018-03-25 | Southampton - Background (near city centre, AURN site) | 29.12083 |
| 2019-04-21 | Southampton - Background (near city centre, AURN site) | 28.97083 |
| 2017-02-14 | Southampton - Background (near city centre, AURN site) | 28.96250 |
| 2019-03-28 | Southampton - Background (near city centre, AURN site) | 28.46250 |
| 2019-04-19 | Southampton - Background (near city centre, AURN site) | 28.36250 |
| 2016-01-19 | Southampton - Background (near city centre, AURN site) | 28.12917 |
| 2017-02-10 | Southampton - Background (near city centre, AURN site) | 27.43333 |
| 2017-08-28 | Southampton - Background (near city centre, AURN site) | 27.42500 |
| 2019-01-04 | Southampton - Background (near city centre, AURN site) | 27.05000 |
| 2018-11-05 | Southampton - Background (near city centre, AURN site) | 26.94167 |
| 2019-02-26 | Southampton - Background (near city centre, AURN site) | 26.87391 |
| 2018-01-12 | Southampton - Background (near city centre, AURN site) | 26.85417 |
| 2018-12-28 | Southampton - Background (near city centre, AURN site) | 26.82500 |
| 2016-05-07 | Southampton - Background (near city centre, AURN site) | 26.77500 |
| 2019-01-21 | Southampton - Background (near city centre, AURN site) | 26.65000 |
| 2019-04-20 | Southampton - Background (near city centre, AURN site) | 26.57917 |
| 2017-06-21 | Southampton - Background (near city centre, AURN site) | 26.56667 |
| 2017-03-28 | Southampton - Background (near city centre, AURN site) | 26.47500 |
| 2019-04-06 | Southampton - Background (near city centre, AURN site) | 26.32917 |
| 2020-01-20 | Southampton - Background (near city centre, AURN site) | 26.30417 |
| 2020-01-22 | Southampton - Background (near city centre, AURN site) | 26.27083 |
| 2018-05-20 | Southampton - Background (near city centre, AURN site) | 26.03333 |
| 2018-05-24 | Southampton - Background (near city centre, AURN site) | 25.93750 |
| 2017-11-01 | Southampton - Background (near city centre, AURN site) | 25.77500 |
| 2016-05-12 | Southampton - Background (near city centre, AURN site) | 25.75000 |
| 2018-04-19 | Southampton - Background (near city centre, AURN site) | 25.75000 |
| 2017-06-20 | Southampton - Background (near city centre, AURN site) | 25.74167 |
| 2018-05-29 | Southampton - Background (near city centre, AURN site) | 25.70833 |
| 2019-12-04 | Southampton - Background (near city centre, AURN site) | 25.51304 |
| 2017-05-06 | Southampton - Background (near city centre, AURN site) | 25.04167 |
Report generated using knitr in RStudio with R version 3.6.3 (2020-02-29) running on x86_64-apple-darwin15.6.0 (Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64).
t <- proc.time() - startTime
elapsed <- t[[3]]
Analysis completed in 74.173 seconds ( 1.24 minutes).
R packages used:
Arino de la Rubia, Eduardo, Hao Zhu, Shannon Ellis, Elin Waring, and Michael Quinn. 2017. Skimr: Skimr. https://github.com/ropenscilabs/skimr.
Dowle, M, A Srinivasan, T Short, S Lianoglou with contributions from R Saporta, and E Antonyan. 2015. Data.table: Extension of Data.frame. https://CRAN.R-project.org/package=data.table.
Grolemund, Garrett, and Hadley Wickham. 2011. “Dates and Times Made Easy with lubridate.” Journal of Statistical Software 40 (3): 1–25. http://www.jstatsoft.org/v40/i03/.
Müller, Kirill. 2017. Here: A Simpler Way to Find Your Files. https://CRAN.R-project.org/package=here.
Sievert, Carson, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, and Pedro Despouy. 2016. Plotly: Create Interactive Web Graphics via ’Plotly.js’. https://CRAN.R-project.org/package=plotly.
Wickham, Hadley. 2009. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. http://ggplot2.org.
Zhu, Hao. 2018. KableExtra: Construct Complex Table with ’Kable’ and Pipe Syntax. https://CRAN.R-project.org/package=kableExtra.